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ABSTRACT 



The report describes research strategies to explore the 
relationship between literacy and employment and to incorporate the influence 
of job skills with California Work Opportunity and Responsibility to Kids 
(CalWORKs) students at Cabrillo College (California) . The overall design 
tests the literacy and job skills of study participants before and after 
completing their program and then conducts a follow-up survey on their 
employment after leaving the program. Literacy would be measured with 
vocational assessments augmented by course and grade history. Job skills 
would be measured with vocational assessments developed locally and by a test 
developer. The employment follow-up would occur through the already existing 
efforts to track employment of vocational students. The study would run for 
five years. Results from this study would provide a detailed picture of the 
academic and vocational success of CalWORKs students in relation to their 
literacy and job skill levels. The data would show what qualities of the 
student most strongly predict employment, which is critical for program 
planners and students to know. For instance, the analysis would reveal 
whether basic skills or job skills differ in their influence on 
employability. The report also presents the results of a pilot study 
conducted to develop methods of attracting participants and to evaluate the 
assessment instruments. (Contains 14 references.) (JA) 
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Summary 

Increases in literacy have been shown to relate to increases in employment. This 
report describes research strategies to further explore this relation and incorporate the 
influence of job skills with CalWORKs students at Cabrillo College. The overall design 
tests the literacy and job skills of study participants before and after completing their 
program and then conducts a follow-up survey on their employment after leaving the 
program. Literacy would be measured with academic assessments augmented by course 
and grade history. Job skills would be measured with vocational assessments developed 
locally and by a test developer. The employment follow up would occur through the 
already existing efforts to track employment of vocational students. The study would run 
for five year. A less resource intensive alternative would use existing records of 
assessments, grades, course history, and employment surveys. This would not be as 
standardized as a controlled study and job skills measures would be more indirect. 
Results would be useful for program planners and policy makers in deciding how to best 
improve skills and employability. 
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Introduction 



Literacy is often viewed as one of the hallmarks of humanity and widespread literacy 
as one of the indicators of a society moving towards egalitarian enlightenment. Not only 
is the ability to create and interpret symbols to convey information an amazing 
consequence of cognition, whether it is recording the number of goats in your herd or 
calculating the density of a quasar, it is also necessary to command a comfortable place in 
an industrial society. For example, one who lacks reading and arithmetic skills cannot 
read and interpret the manual to operate new software or equipment, write a memo or 
report for your colleagues, or tally expenditures. Those with basic literacy skills can be 
expected to have more life options. 

Before the virtues of literacy are further exalted, a precise characterization should be 
presented. The 1992 National Adult Literacy Survey defined literacy as “using printed 
and written information to function in society, to achieve one’s goals, and to develop 
one’s knowledge and potential” while the National Literacy Act (NLA) of 1991 defined 
literacy as “an individual’s ability to read, write, and speak in English and compute and 
solve problems at levels of proficiency necessary to function on the job and in society, to 
achieve one’s goal, and to develop one’s knowledge and potential (Kirsch et al. 1993).” 
These definitions are similar in that they indulge in a broad interpretation of literacy 
beyond symbol recognition and manipulation to using these abstractions to affect action. 
Some may view the NLA’s specification of English as overly restrictive for a universal 
definition and may have had political motivations but probably stands as a reasonable 
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definition within the United States. It should be noted that literacy definitions include 
quantitative skills as well as verbal abilities. 

Several studies have shown a positive association between literacy and employment 
(Johnson and Trappe 1992, Rivera-Batiz 1992, Kirsch et al. 1993, Brooks and Buckner 
1996) or at least an association between illiteracy and unemployment (Malicky and 
Norman 1994). For instance, the National Adult Literacy Survey (Kirsch et al. 1993) 
found that about half of all U.S. American adults fall in the lowest two of the five levels 
of literacy defined in the study and that 4% of the total were totally illiterate. These 
findings showed a decline in literacy from the 1985 literacy survey. The study 
demonstrated positive relations between literacy level and education, employment, and 
income. Those with low literacy were more likely to receive food stamps and less likely 
to vote. Older adults tended to have lower education and literacy levels and those from 
non-white ethnic groups tended to have lower education and literacy levels and were 
more likely to be immigrants. 

Such results have implications for those interested in moving some of those on public 
assistance to employment. The United States’ initiated Aid to Families with Dependent 
Children (AFDC) in 1935 (Withom 1996, HHS 2000), which provided cash assistance to 
low income families. Its existence was haunted by philosophical ambivalence with the 
desire to help those in need in opposition to a distaste for providing money in the absence 
of work (Withom 1996, Berlin 2000). This lead to a number of reforms and revisions 
until the program was terminated in 1996 with the passage of the Personal Responsibility 
and Work Opportunity Reconciliation Act (PRWORA) (Berlin 2000, HHS 2000). In 
replacement, each State was given a block grant from Temporary Assistance to Needy 
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Families (TANF) funds to allocate within broad constraints. One of these was the 
requirement that recipients be engaged in 32 hours per week of work or work preparation 
activities with an 18 month time limit upon first receiving assistance and a 60 month 
lifetime limit. The intent was to encourage States to create programs that quickly moved 
recipients off assistance and into employment and required work related activities in 
exchange for aid. In addition, some hoped that welfare reform would result in increased 
employment and a savings in public welfare expenditures (Moffitt 1996). While at least 
one early study cast doubt upon this potential for “workfare” programs (Grossman and 
Roberts 1989) more recent studies indicate that obligatory participation in employment 
related activities could increase employment (Friedlander and Hamilton 1996, Seniger 
1998, Berlin 2000). Friedlander and Hamilton (1996) found that San Diego’s Saturation 
Work Initiative Model (SWIM) resulted in higher employment for participants and a 
savings to taxpayer’s but the structure of the program did not result in an increase in 
participants’ income. Newer programs reviewed by Berlin (2000) showed that programs 
structured to create work incentives by allowing retained earnings to increase faster than 
benefit reductions increased employment and participants’ incomes, thereby reducing 
poverty, but at a slight increase in expenditures over the short term. 

California’s program, California Work Opportunity and Responsibility to Kids 
(CalWORKs), began shortly after AFDC ended and allows each County wide latitude in 
the implementation of CalWORKs. Consistent with findings that increased literacy as 
well as job skills increases employment opportunities, Santa Cruz County allows 
enrollment at Cabrillo Community College to act as a work preparation activity. In this 
project, we are interested in designing a protocol to precisely describe the relationship 
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between literacy, job skills, and employment and income of Cabrillo College CalWORKs 
students. 

Methods 

The overall design tests the literacy and job skills of study participants before and 
after completing their program and then conducts a follow-up survey on their 
employment after leaving the program. Outcome measures would include program 
success rate, employment, employment in the field of preparation, income, quality of 
employment (i.e. does the job provide benefits etc.), and job satisfaction. Predictors 
would consist of pre- and post-program literacy and job skills, increase in skills, and 
demographic characteristics such as GPA and ethnicity. 

Pilot Study Methods 

A pilot study was implemented to develop methods of attracting participants and to 
evaluate job skills assessment instruments. Participants were required to take Cabrillo’s 
math and reading assessments, WorkKeys observation and listening/writing assessments, 
a faculty created vocational exam within their program, and to edit the employment 
follow-up survey. 

Cabrillo’s current math assessment instrument and the one used in the pilot study is 
the Mathematics Diagnostic Testing Project (MDTP). The Math Department will 
probably switch to the COMPASS test by ACT, a non-profit test development group, by 
2001. Cabrillo’s reading assessment instrument is ACCUPLACER by College Board. 



The WorkKeys assessments are designed by ACT and are intended to test 
employment relevant skills generalizable to all fields of work. The observation test 
presented examinees with a video of segments portraying workers at various jobs being 
trained in or performing work tasks correctly. This is followed by several segments 
showing the worker performing tasks correctly or incorrectly and examinees were asked 
to identify which step if any was in error or to note a particular detail or to select the next 
step in the process. The listening/writing test presented examinees with an audio tape of 
simulated work related messages left on an answering machine. Examinees were asked 
to take notes based on the message and compose a cogent memo. Listening and writing 
each received a separate score. 

Three faculty from three vocational programs, Business, Culinary Arts and 
Hospitality Management, and Medical Assistant, were compensated to construct a 
comprehensive exam for students in their program for use as a pre- and post-test. The 
faculty constructed vocational exams were all multiple-choice comprehensive tests that 
took about 20 to 45 minutes to complete. The employment survey was a modification of 
the instrument used for Cabrillo’s vocational follow-up study. Drafts of this survey and 
the surveying protocol were commented on by members of the LADDERS project, which 
is a consortium of adult services such as the Human Resources Agency, Employment 
Development Department, Cabrillo’s Fast Track office, Cabrillo’s Occupational 
Education program. Shoreline employment services, ROP, and Santa Cruz Adult 
Education. These groups’ interest in this study stems from their desire to implement 
employment follow-up surveys that are consistent between all groups and also meet State 
reporting requirements to the extent possible. They have each expressed interest in this 
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study and indicated they would be willing to assist with study design, implementation, 
and data sharing within constraints. 

For the first round of testing, faculty announced the study opportunity in their classes 
informing students of the nature of the study and that they would be compensated with a 
$100 gift certificate to Staff of Life, a local heath food market that was willing to 
participate. While CalWORKs students were the focus, non-CalWORKs students were 
also allowed to participate. The first effort resulted in a total of 4 students comprised of 1 
CalWORKs Medical Assistant student, and 1 non-CalWORKs Medical Assistant 
students, and 2 non-CalWORKs Culinary Arts students. No Business students 
participated in the first round. 

For the second round of testing, contact information from the Data Warehouse was 
used to compile a list of all students that were enrolled in the three vocational programs. 
Their CalWORKs status was determined with help from the Fast Track office. The 
names of the students were then separated into two categories, CalWORKs and non- 
CalWORKs. Names within each category were then randomized. The list containing 
CalWORKs students were called first followed by the non-CalWORKs students. Each 
possible participant contacted was given a short overview of the pilot study, were 
informed that all information was confidential, and were told that they would receive a 
gift certificate for their participation in the study. 

Researchers called 21 CalWORKs students with a total of 8 (38%) willing to 
participate and 38 non-CalWORKs students with 5 (13%) willing to participate. When 
talking with potential students, it appeared that CalWORKs students tended to show more 
interest in the study and seemed to have a more flexible schedule compared to non- 
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CalWORKs students, however, childcare was a major constraint. In the non- CalWORKs 
students contacted, work was consistently a constraint. Also, the gift certificates given as 
compensation appeared to be more attractive to CalWORKs students than non- 
CalWORKs students. 

By the end of the pilot study, 14 students had participated. All were females between 
the ages of 24 and 52. 64% identified as White, and 2 as Hispanic, 1 as African- 
American, 1 as Native American, and 1 as Other. The ethnicity distribution was similar 
to that for the student population. Table 1 shows the program affiliation of participants 
by CalWORKs status. 

Table 1. Pilot study participants by program affiliation and CalWORKs status. 





Medical Assistant 


Business 


Culinary Arts 


CalWORKs 


5 


2 


2 


Non-CalWORKs 


1 


1 


3 



Pilot Study Results and Discussion 

Faculty announcements were highly variable between programs in their effectiveness 
and direct phone solicitation did not yield participation rates needed for a larger, longer 
term study. Part of the difficulty with attracting participants in the pilot study was the 
brief window of opportunity to participate. A long-term institutionalized study with 
involvement by student service offices should yield much higher participation rates. 
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The WorkKeys listening scores showed a significant and strong positive linear 
correlation with the reading scores (r = 0.86, p < 0.0005), the observation scores (r = 
0.82, p = 0.001), and GPA (r = 0.84, p = 0.009). For example, with reading scores 
showed no overlap between the two levels of listening scores (Figure 1). 




Listening Score 

Figure 1 . Relation between WorkKeys listening score and ACCUPLACER Reading 
score among pilot study participants. 

This suggests that other tests and measures capture much of what is being tested in the 
listening/writing test for the participants in this pilot study. In addition, Cabrillo’s 
English assessment already contains a writing section. It does not appear that the 
WorkKeys listening/writing test would provide enough additional information in relation 
to its high cost ($16 per unit) to warrant its inclusion in the full scale study. 



The WorkKeys observation test showed a significant and moderately strong positive 
linear correlation with the reading scores (r = 0.69, 0.009) and listen/writing as noted 
above. This test seems to be more unique in what it measures and more closely 
resembles the visual and aural learning that accompanies most jobs. Its lower cost ($6 
per unit) and lack of resemblance to academic assessments argues for its inclusion in the 
full-scale study. 

The faculty-constructed exams were only taken by 3 to 6 students and so definitive 
evaluations of these tests are not currently possible. Most of the questions appeared 
reasonable to the students although further refinements could be made. For the full-scale 
study, teams of faculty from each program would construct these tests, which would be 
validated and improved through the study. 

Full Study Methods 

This study intends to focus on Cabrillo students who are CalWORKs participants. A 
CalWORKs student is a student who is in CalWORKs at enrollment or is in CalWORKs 
at least once during enrollment. The ideal student is one who is in CalWORKs at the 
beginning of a program and remains on until receipt of their degree or certificate. These 
students would be identified with help from Cabrillo’s Fast Track office, a one-stop 
social service center for CalWORKs students. 

Students enrolled in selected vocational programs will be targeted for participation. 
Depending upon faculty participation, these will probably include Business, Computer 
Science, Culinary Arts, Early Childhood Education, and Medical Assistant. These 
programs are attractive due to their large number of students and/or faculty interest. As a 
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group, they capture a broad array of job types from the quantitative to the verbal and 
from a service to a production orientation. If funding permits, more vocational programs 
will be included. 

Students’ declared majors at enrollment will initially identify students in these 
programs. Their final program affiliation will be determined by the field in which they 
received their program award or by the program in which they completed the most units 
if they leave before completing. Information on students will be derived from Cabrillo’s 
Data Warehouse compiled by Computing Resources and analyzed by the Planning and 
Research Office. 

We will attempt to involve all CalWORKs participants within selected vocational 
programs by institutionalizing the study. Literacy and job skill testing will be linked to 
the Assessment Center and the Fast Track office. To the extent funding permits, extra 
testing will result in non-cash compensation, such as gift certificates, to avoid affecting 
participants’ benefits. As was shown in the section on the pilot test, other methods of 
attracting participants did not result in adequate success rates. By institutionalizing 
participation, the default choice for students would be to participate with compensation. 
Those who did not wish to participate would be encouraged to involve themselves to help 
improve programs for future students. However, no student would be unduly compelled 
to participate or denied access to courses based upon participation status. 

Cabrillo’s reading, writing, and mathematics assessment tests most conveniently 
measure literacy levels. These tests have already been validated and must be taken by 
most students anyway. To ensure comparability of scores, all students will take at least 
the most basic version of each test where applicable. A written protocol outlining this 
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requirement will be given to each participant to give to Assessment Center workers who 
will also be briefed verbally and in writing by the researcher. 



Job skills are not routinely tested for at Cabrillo and would require implementing a 
new set of instruments. Job skills include general skills such as observation and listening 
skills, punctuality, and interpersonal skills. Others are more specific to a given 
occupation such as knowing how boiling broccoli in acidic water affects the color. For 
general job skills, we will use a WorkKeys assessment by ACT. One of their promising 
tests is an observation test that could be administered in addition to Cabrillo ’s academic 
assessments. This test would round out the academic tests and is much less costly to 
implement than some other alternatives. One of these is the CASAS assessment. An 
advantage of CASAS is that Santa Cruz County’s Human Resource Agency uses this test, 
which would aid in comparing data and potentially reducing assessment duplication. The 
main disadvantage is the high monetary and labor cost as it requires a substantial 
institutional commitment to employee training to implement the CASAS program. 

Vocation specific skills would be assessed by comprehensive exams developed by 
faculty within each participating vocational program. These tests would contain 
questions designed to test knowledge directly related to the vocational program. In some 
cases, such as for Culinary Arts, these will also include skills demonstrations. Overall 
GPA of vocational courses will serve as an additional measure of vocational skills. This 
is desirable as testing alone captures test taking ability to an unknown degree while 
course grades often incorporate factors important to employment success such as 
participation, punctuality, ability to follow instructions, and task completion. Other 
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proxy measures of job skills would include job experience as determined by study 
participant self-report. 

Employment outcomes of study participants will be measured during Cabrillo’s 
annual Vocational Program Completer and Leaver Survey. Each year, those who have 
completed a vocational program (Completers) and those who took at least 10 units in a 
program before not re-enrolling for a year (Leavers) are identified through Cabrillo’s 
Data Warehouse. Students are contacted until either they respond or two mail and three 
phone contact attempts have been made. Contacted students are asked to reply to a 
survey (see attachments) that requests information that includes employment status, 
income, whether the job has benefits, and job satisfaction in addition to other questions 
designed to aid program planning. As an incentive, survey completers are told they are 
placed into a drawing for a $50 prize, which is given at the end of the data collection 
cycle. Historically, this effort has resulted in an almost 40% return rate. For the 
purposes of this study, CalWORKs participants would be included in the vocational 
follow-up every year for several years to track employment retention. As this follow-up 
survey already occurs, it will reduce the funding burden on the full study. 

Predictors will be related to outcome measures using logistic and multivariate 
regression to predict probability of employment and level of income based upon 
participant characteristics (Enberg, Gottschalk, and Wolf 1990, Seninger 1998). A cohort 
of participants would probably be available the first year after the study begins although 
some may be lacking systematic pre- and post-test scores. Subsequent years will see 
increases in the numbers of program completers and leavers with pre- and post-test scores 
available for study. 
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The study should run a minimum of 5 years to allow for participants to enter Cabrillo, 
be pre- and post-tested, complete their programs, and seek and keep employment. In the 
first year or two, post-tests would be given to exiting students and their employment 
tracked. This would still allow for analysis of the relation between exit skills and 
employability but would miss information about skills gains during their time at Cabrillo. 

Alternative Design: Archival Study 

The institutionalized research protocol would require a significant commitment of 
resources from the college and many faculty and staff The organization to conduct the 
study would not only have to be extensive, it would also have to persist in the face of 
changes in funding and turnovers in involved staff and faculty. 

An alternative approach would involve using existing data to examine the relations 
between literacy and employment. Students at Cabrillo already have their academic 
performance recorded, including assessments or placement tests. Pre-completion literacy 
could be indicated by their English, Reading, and Math placement scores prior to taking 
coursework and post-completion literacy could be shown by highest level of English and 
Math completed with grades. A measure of level of job skills could be overall GPA of 
vocational classes in their field or grades in key courses, as determined by faculty. 
Employment follow-ups are already conducted for all Vocational students at Cabrillo and 
would provide information on employment and income. With help from the Fast Track 
office, students could be identified as CalWORKs participants so that we could focus on 
those students and compare them with non-CalWORKs students. The Human Resources 
Agency could be asked to aid in identifying CalWORKs participants and to provide data 
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on assessments and follow-ups that they conduct. This study could expand to ask other 
employment training providers such as Adult Education and Shoreline to share their data, 
which includes employment follow-up data. The challenge of including other trainers 
would be to equate program performance measures to assess skill levels. 

Some of the savings from the archival approach would be invested in increasing the 
employment follow-up survey response rate and its representativeness. Strategies include 
increasing the amount or number of incentives given to respondents and making 
multilingual mail surveys and having multilingual phone surveyers as well as increasing 
coordination with other agencies to maintain files of current contact information for 
students. 

This archival approach has the advantages of being able to be done in a relatively 
brief period, requiring fewer persons to execute, and having a smaller funding footprint. 

A principal researcher with the aid of assistants would compile and analyze the data with 
occasional consultation by faculty and human resource and employment training staff. A 
disadvantage would be the reduction in standardization of skill measures. Students 
would take assessments at various points in their career despite that fact that all new 
students are encouraged to assess in their first semester. This problem would compound 
if the study used data from other training programs. 

The measures of job skills would be less direct as well. Rather than testing job skills 
with instruments specifically designed for this purpose, we would rely on grades in 
vocational courses to capture information about knowledge and skill level in that field as 
well as general skills such as participation and timeliness of completing assignments. 
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Despite these concerns, this alternative approach should provide enough useful data to 
illuminate the relation between literacy, job skills, and employability. 

Dissemination of Results 

The principal researcher will prepare annual progress reports and a final report at the 
end of the study period with assistance from participating faculty, staff, administrators, 
and stakeholders in other agencies. These reports would be distributed in print form on 
100% post-consumer waste recycled paper to interested parties such as study funders. 
State and Federal policy makers, program planners, and study participants. The reports 
would be made globally available on the Planning and Research Offices website in PDF 
format. Links to this report from other agencies’ websites would be requested. In 
addition, a paper would be submitted for publication to a peer-reviewed journal and for 
presentation at professional conferences. Finally, press releases of findings after 
publication would be sent to local and national media outlets. 

Importance 

Results from this study would provide a detailed picture of the academic and 
vocational success of CalWORKs students in relation to their literacy and job skill levels 
and demographic characteristics. Primarily, the data would show what qualities of the 
student most strongly predict employment. For instance, the analysis would reveal 
whether basic skills or job skills differ in their influence on employability. This is critical 
as CalWORKs students are operating under a very limited time budget and program 
planners and students need to know which skills they should focus on to meet career 
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goals. In addition, policy makers would benefit from seeing if the time limit is sufficient 
to acquire enough skills to improve employability. 

If the currently proposed employment follow-up data is shared among members of the 
LADDERS project, cross program comparisons can also made. Since assignment to 
programs is not random, comparisons would be qualified but should still provide useful 
data especially if comparison groups are taken from within the same geographic location 
at the same point in time (Friedlander and Robins 1995). 

Funding 

Some of the study costs would be absorbed by Cabrillo’s normal operating budget 
such as academic assessment tests, at least part of the employment follow-up costs, and 
compiling demographic and academic data on participants. Other expenses would 
require additional funds such as extra teaching units for participating faculty, salary 
offsets for involved administrators and staff, compensation for the researcher, 
consultants, and participants, payments for vocational test units, copying and printing, 
data entry, supplies and equipment, and other items and services as needed. 
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